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ABSTRACT 



This study investigated the use of asynchronous (mailing 
lists) and synchronous (chat sessions) Internet-based communication and their 
impact on teachers’ attitude toward collaboration, activity completion rates, 
and test performance. The study also investigated the impact of collaboration 
on activity completion rates and teacher performance measured by objective 
tests. The treatment was an interactive web-based course designed to enhance 
the collaborative teaching and learning communities of participating 
educators as they acquired Internet skills. Participants had access to chats 
and were members of two course-related mailing lists. They completed a 
pretest that assessed their Internet skills in the areas of searching and 
using e-mail. Following the pretest, they completed tutorials specific to 
these skill areas; then they completed a posttest. The study used two 
instruments, the "Stages of Concern Instrument: Attitudes Toward the 
Internet" and a criterion-reference performance test. Researchers logged use 
of mailing lists and Internet chats to measure engagement in Internet-based 
communication as a collaborative tool. Data analysis indicated that attitudes 
toward collaboration did not affect test performance, though there was a 
relationship between attitudes toward collaboration and use of Internet-based 
communication. (Contains 17 references.) (SM) 
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Abstract 

This study investigated the use of asynchronous (mailing lists) and synchronous (chat sessions) 
Internet-based communication and its impact on teachers’ attitude toward collaboration, activity 
completion rate and test performance. In addition, the impact of collaboration on activity 
completion rate and teacher performance measured by objective tests was investigated. Although 
it was found that attitudes toward collaboration did not affect test performance, the data 
suggested a relationship between attitudes toward collaboration and use of Internet-based 
communication. 
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Impact of asynchronous and synchronous Internet -based communication on collaboration and 

performance among K-12 teachers 

Objectives 

Cognitive models of learning stress that learners are active agents in constructing their 
own learning (Shaw, 1995; Biehler & Snowman, 1986). Collaborative learning enhances the 
learning process, allowing learners to engage in the construction of new representations of 
knowledge and facilitating the transfer of information from short-term to long-term memory 
(Hooper, Temiyakam, & Williams, 1994; Alavi, 1994). Asynchronous and synchronous 
communication provides users with opportunities to collaborate across distances through such 
technologies as mailing lists and chat sessions, respectively. It was hypothesized that these 
modes of communication provide unique features conducive to the development of a 
collaborative community of learners. The benefits of Internet-based communication and 
collaboration are not reflected solely in test performance. Behind the behavioral change is an 
attitudinal change. Gain in test performance might be short term; change in attitude could 
facilitate life-long learning (Yu, 1998). Therefore, this study investigated the use of asynchronous 
(mailing lists) and synchronous (chat sessions) Internet-based communication and its impact on 
teachers’ attitude toward collaboration, activity completion rate and test performance. In 
addition, the impact of collaboration on activity completion rate and teacher performance 
measured by objective tests was investigated. 

Theoretical Framework 

Several theoretical perspectives suggest that peer interaction occurring in computer 
networked groups may be beneficial to learning. These perspectives include Vygotsky's (1978) 
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and Piaget's (1963) emphasis on social construction of knowledge and Bandura's (1969) 
interpretation of peer modeling (King, 1989). From a social cognition point of view, when 
educators interact with peers they are faced with ideas, explanations and information inconsistent 
with prior knowledge and beliefs. Confronting these inconsistencies challenges one's current 
point of view; resolving such cognitive conflicts results in cognitive restructuring (Bearison, 
1982). Group members presumably add detail to existing cognitive schemas, explore and correct 
misunderstandings, fill in gaps in their knowledge, and/or reorganize their knowledge structures 
(Bearison, 1982; Piaget, 1963; Vygotsky, 1978). During peer interaction, educators are exposed 
to new strategies, terminology, and ways of thinking about problems; through restructuring 
thinking, these experiences affect problem-solving behaviors as well as classroom instructional 
delivery. From a social modeling perspective, peer interaction provides opportunities for 
imitation of successful problem-solving behaviors. Thus changes in teachers' levels of 
competence take place as a result of peer interaction in small groups (Vygotsky, 1978). 

Collaborative learning is the acquisition by individuals of knowledge, skills or attitudes 
as a result of group interaction, where that specific learning could not have been derived 
individually (Schrage, 1990). Computer-mediated collaboration provides a medium for educators 
to learn to engage in collaboration with other educators at a distance. Alavi (1994) observes that 
those involved in collaboration can monitor individual thinking, opinions and beliefs and provide 
feedback for clarification and change. In helping one another, learners may foster stronger 
elaborations between new and existing knowledge, facilitating the transfer of information from 
short-term to long-term memory (Hooper, Temiyakam, & Williams, 1994). 

Internet-based communication provides both asynchronous and synchronous 
environments for collaboration. Synchronous communication (such as chat) allows interaction to 
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occur at the same time among learners, as in classroom discussions: but learners in the Internet 
environment use a keyboard to type messages. Local Internet accounts enable learners to engage 
in collaborative chat sessions with distant peers, expanding the collaborative environment from a 
single classroom to classrooms around the world. Asynchronous communication (mailing lists) 
allows interaction to occur at different times and locations between two or more learners. 

Learners need not be present to receive information, and may communicate when ready. Learning 
occurring as a result of group interaction may take place in a virtual community and may be 
adjusted to student need, schedule and educational goals. Both of these forms of Internet-based 
communication expand learners' access to information, resources, and collaboration, and may 
increase learners' attitude toward collaboration as well as achievement performance. In this study, 
we are interested in both forms of Internet-based communication as they relate to learner attitude 
toward collaboration and the completion rate and performance on Internet skill tests. 

H}'potheses 

Based on the above, the following h}'potheses were formed: 

(a) Use of Internet-based communication increases teachers’ positive attitude toward 
collaboration on the Internet as measured by the Stages of Concern Instrument: Attitudes 
Toward the Internet (ATT, Hall, George, & Rutherford, 1977; Well and Anderson, 1997); 

(b) Use of Internet-based communication increases course completion rate; 

(c) Use of Internet-based communication increases test performance as measured by criterion- 
referenced Internet skills test; 

(d) The more positive teachers’ attitudes toward collaboration, the higher the increase in teacher 
performance as measured by criterion-referenced Internet skills test; and 
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(e) The more positive teachers’ attitudes toward collaboration, the higher the course completion 
rate. 

Methods 

Participants 

Participants in this study were one hundred sixty-one K-12 educators residing throughout 
a southwestern state. All participants took part in an interactive web-based course designed to 
enhance the collaborative teaching and learning communities for participating educators as they 
acquired Internet skills. 

Treatment 

The treatment was an interactive web-based course designed to enhance the collaborative 
teaching and learning communities for participating educators as they acquired Internet skills. Of 
particular interest were new outcomes made possible through use of advanced technologies in 
instruction. Therefore we looked at the engagement in Internet-based communication and 
increases in Internet skills. 

Procedure 

Participants were part of a large collaborative community (the course) as well as of 
smaller communities focused on establishing professional relationships and on technical and 
course help. These smaller communities were facilitated by a group leader trained in the use of 
Internet technologies and in how to facilitate Internet-based communication. Group leaders were 
not participants in the course. Participants had access to chats and were members of two course- 
related mailing lists, (a) course announcements, and (b) a group list headed by the group leader. 
Group leaders conducted at least one chat bimonthly and communicated with their participants 
via group mailing lists biweekly. Participation in chats was mandatory for group leaders and 
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Strongly encouraged for participants. Participants were encouraged to join other mailing lists 
within the course setting as well as outside of the course. The course itself maintained up to 25 
mailing lists on various content-specific topics. Archives of eleven chat sessions and eleven 
mailing list groups were used in this investigation. 

The Internet skills that participants were to learn focused on searching techniques and the 
use of email. Participants completed a pretest assessing their Internet skills in these areas. 
Following the pretest, participants completed tutorials specific to these skill areas, then 
completed a posttest. 

Data collection 

Two instruments were employed in this study. The first instrument was the Stages of 
Concern Instrument: Attitudes Toward the Internet (ATI; Hall, George, & Rutherford, 1977). 

This scale was developed to measure the evolving attitudes of learners exposed to an innovation. 
In this original scale, the variable word “irmovation” may be substituted with the irmovation of 
interest. Therefore, in a study regarding learner attitudes toward a new irmovation, specifically 
the Internet, Wells and Anderson (1997) substituted the variable “irmovation” with “Internet” to 
measure attitudes toward the Internet. Reliability of the instrument was reported as .91 (Reed, 
1990), and discussed in detail by Wells and Anderson (1997). The second instrument was a 
performance test over the Internet skills learned that was developed by our research team. For 
both measurements, a pretest was administered prior to a web-based instruction program. Three 
months later a posttest was given. The interval was short in order to avoid the "maturation" effect 
in which subjects improve over time regardless of treatments (Cook & Campbell, 1979). All 
participants completed the pretest and 95 finished the posttest. One hundred and fifty-nine 
participants completed the ATI survey and fifty-six completed the post-survey. 
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A subscale of ATI was used to measure the attitude toward collaboration though the 
entire survey was administered to the participants. The survey was composed of 35 questions and 
could be divided into seven subscales, which indicated seven stages of concern with the Internet. 
The seven stages were awareness, informational, personal, management, consequence, 
collaboration, and refocusing (Wells & Anderson, 1997). Please see Table 1 for the items in the 
collaboration subscale. 

Table 1. 

Collaborative subscale in Stages of Concern Instrument: Attitude Toward Using the Internet 



Question 5: 
Question 10: 
Question 18: 

Question 27: 
Question 29: 



I would like to help other faculty use the Internet. 

I would like to work with present fellow workers and others who are using the Internet. 

I would like to familiarize my colleagues about the Internet as I learn more about it and with it 
more. 

I would like to coordinate my efforts in learning about the Internet with fellow workers. 

I would like to know what other people are doing in relation to the Internet. 



Factor analysis confirmed the factor structure of the instrument. Based on the pre-survey 
data, the survey items could be grouped as eight latent variables according to the Kaiser criterion 
(eigenvalue as one or larger). By examining the inflection point in the scree plot, seven factors 
could extract most of the eigenvalues (Figure la). In the post-survey, Kaiser criterion also 
suggested eight factors while the scree plot implied five to seven factors (Figure lb). 



Internet-based communication 9 



Figure 1 (a). Factor extraction by Kaiser criterion and the inflection point in the scree plot for 
pre-survey. 
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Figure Ub). Factor extraction by Kaiser criterion and the inflection point in the scree plot for 
post-survey. 




Based upon the pre-survey data, the estimated reliability of the entire ATI survey in terms 
of Cronbach coefficient Alpha was .86 whereas the Cronbach coefficient Alpha for the 
collaboration subscale was .87. In the ATI post-survey data, the overall Alpha was .89 while the 
collaboration subscale Alpha was .79. 

The second instrument in this study was a criterion-reference performance test, which 
consisted of 30 items. Cronbach coefficient Alpha of the pretest was .45 whereas that of the 
posttest was .86. The low Alpha in the pretest, which was expected, implied that participants 
were unfamiliar with the subject matter and random guessing of answers led to inconsistency. In 
the posttest the high Alpha denoted that the response pattern was internally consistent. 

Finally, use of mailing lists and Internet chat was logged as a measure of engagement in 
Internet-based communication as a collaborative tool. Originally, engagement in collaboration 
was defined by the number of words in email/chat and the total number of messages. However, 
these distributions were highly skewed and thus violated the assumptions of parametric tests. 
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Therefore, observations were re-coded into four binary variables: (a) use of mailing lists, (b) use 
of chat sessions, (c) use of both mailing lists and chat sessions, and (d) use of neither mailing 
lists or chat sessions. For example, if the participant used mailing lists, the value of the first 
preceding variable was re-coded as “1” regardless of either the number of words in the posted 
message or the number of messages. If no messages were posted, the value was recorded as “0”. 
The remaining variables were similarly re-coded. 

Results 

Descriptive statistics are reported in Table 2. Pearson Product Moment correlation 
coefficient between the Internet skills pretest and posttest scores was .13 whereas that of the ATI 
pre-survey and post-survey was .29. 

Table 2 

Descriptive statistics for Internet skills test and Attitude Toward the Internet. 



Variable 


M 




n 


Internet Skills 


Pretest 


12.83 


3.31 


161 


Posttest 


22.48 


4.88 


95 


Difference Score 


9.12 


5.45 


95 


Collaboration Subscale 


Pretest 


29.14 


5.27 


159 


Posttest 


29.09 


4.32 


56 


Difference Score 


-.67 


5.04 


43 



Results of the Kolmogorov-Smimov Normality tests indicated that the distribution of the 
Internet skills pretest scores did not significantly depart from normality, w (174) = 97, p = .09. 
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Similarly, neither did the difference between the pretest score and the posttest score, w (94) = 
.97, p = .14. The distribution of the posttest score was not normal, w (94) = .92, p = .0001 . No 
outliers were found in the preceding distributions, which is displayed in Figure 2(a)-2(c). 



Figure 2(a). Distribution of pretest scores on Internet skills. 
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Figure 2(b). Distribution of posttest scores on Internet skills. 
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Figure 2(c). Distribution of difference between Internet skills pretest and posttest scores. 




Results of Kolmogorov-Smimov tests indicated that both distributions of the 
collaboration pre-survey (w (158) = 85, g = .0001) and post-survey (w (55) = .93, g = .0032) 
departed from normality. Non-normality was not observed in the distribution of the difference 
between the pre-survey and the post-survey, w (42) = .95, g = .13. Again, no outliers were found 
in the preceding distributions, which is shown in Figure 3(a) - Figure 3(c). 

Figure 3(a). Distribution of pre-survey on attitude toward collaboration. 
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Figure 3(b). Distribution of post-survey on attitude toward collaboration. 




Figure 3(c). Distribution of difference between collaboration pre-survey and post-survey. 




Prior to addressing the hypotheses, two one-tailed dependent t-tests were employed to 
examine whether improvement in teacher scores on the Internet skills test and the ATI 
collaboration subscale occurred after the treatment. The dependent t-test for the difference 
between the pretest and posttest yielded a significant difference, t (94) = 16.33, p = .0001. Given 
that the effect size was .03, the alpha level was .05, and the sample size was 95, the power for 
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this t-test was .90, which indicated a high probability that the null hypothesis was correctly 
rejected. 

On the average participants had a decrease in the positive attitude toward collaboration 
after the treatment, M = -0.67. A dependent t-test showed that this decrease was not significant, t 
(42) = -0.88, p = 0.38. This result might be due to the lack of statistical power. Given that the 
effect size was .03, the alpha level was .05, and the sample size was 43, the power for this t-test 
was .61, which indicated that a true significant difference might be undetected. 

To test the hypothesis regarding the relationship between engagement in Intemet-based 
communication and attitude toward collaboration (hypothesis a), exploratory data analysis 
instead of ANOVA was performed to examine the relationships. The data structure did not 
conform to the parametric assumptions of ANOVA such as normality and homogeneity of 
variances, and this sample size was not large enough to run a robust ANOVA test. In the 
exploratory data analysis, it was found that users of chat and mail tended to center around the 
mean of the difference between collaboration pre-survey and post-survey, as shown in Figure 4. 
Participants who engaged in chat sessions had less variability on the change score of ATI (-7.5 to 
1.5) than the general test population. Likewise, participants who posted messages on mailing 
lists also centered around the mean (-4.5 to 4.5). Therefore this might indicate that participants 
who engaged in chat sessions or mailing lists have less change in their attitude toward 
collaboration than those in the general test population, maintaining scores close to the mean. 




16 



Internet-based communication 16 



Figure 4, Linked observations between use of collaboration and changed scores in attitude toward 



collaboration 





A Chi-square test of goodness of fit was applied to investigate whether engagement in 
Internet-based communication could increase the likelihood of completing the entire instructional 
module (hypothesis b). Likelihood ratio Chi-square was used because it is related to log-linear 
model and logistic regression model, and completion of module was a dichotomous variable. 

The Chi-square analysis yielded a significant result, Likelihood ratio X = 8.19, p = .042. The 
frequency of each cell is displayed in Table 3. Because one of the cells had no subjects and some 
cell sizes were very uneven, a Chi-square test might not be valid. To rectify this shortcoming, an 
exact test was performed using StatXact (SPSS Inc., 1999). Both asymptotic inference and exact 
inference yielded significant results. In the former the one-sided p-value was .02 whereas in the 
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latter the one-sided p-value was .03. These results indicate that engaging in Internet-based 
communication increases the likelihood of completing Internet activities. 



Table 3 

Frequency table of module completion by use of Internet-based communication . 



Frequency 


Chat 


Mail 


Chat - 
Mail 


None 


Total 


Did not completed the module 


7 


5 


0 


54 


66 


Completed the module 


12 


14 


5 


64 


95 


Total 


19 


19 


5 


118 


161 



As the data structure did not conform to the parametric assumptions of ANOVA, a one- 
tailed two-sample independent t-test was applied to examine hypothesis (c), regarding whether 
engagement in Internet-based communication could lead to improvement as measured by the 
difference between the pretest and the posttest. Originally, use of Internet-based communication 
was classified into four groups, namely, use of chat, use of email, both, and none. For conducting 
a t-test, the first three groups were collapsed into one group. As a result, only two groups 
remained, namely, use of collaboration and no use of collaboration. The two-sample t-test did not 
yield a significant result, t(28) — .61, p = .54. Therefore, use of collaborative tools did not appear 
to have an impact on test performance. 

Regression analysis was adopted to test hypothesis (d), which was concerned with the 
relationship between attitude toward collaboration and test performance. The assumption of 
random residual was checked by plotting predicted scores against residuals (see Figure 5a). The 
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regression analysis disconfirmed use of Internet-based communication as a predictor to Internet 
skills test performance, R (42) = .0005, g = .88. The scatterplot of the two variables did not 
display a pattern (Figure 5b). 



Figure 5(a) . Residuals plotted against predicted posttest scores. 
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Figure 5(b). Scatterplot of attitude toward collaboration against posttest scores. 




3 

O 

•n 

3 

O 

(0 

21 

S5" 



15 - 

1 1 1 1 

20 25 30 35 

Post-survey of attitude toward collaboration 



Internet-based communication 1 9 



A logistic regression was employed to examine hypothesis (e), which addressed the 
relationship between activity completion and the attitude toward collaboration in the Internet. 

The logistic regression analysis did not yield a significant result, p = .60, odds ratio = .98, 
therefore scores on attitude toward collaboration did not predict activity completion rate. 

Discussion 

Collaboration provides a way for educators to form a virtual community of learners in 
which members can engage in individual thinking, share opinions and beliefs and provide one 
other with feedback for professional and personal growth and change. In the course studied. 
Internet-based communication allowed participants to engage in collaboration with others across 
the state. Although the Internet provides different types of environments for communication and 
collaboration among users, their effects are as yet unknown. In this study, it was hypothesized 
that asynchronous and synchronous Internet-based communication provides unique features that 
promote the development of collaborative communities. In addition, this environment would 
then impact participants’ attitudes toward collaboration and performance on Internet skill tests. 

Although in this study it was found that attitudes toward collaboration did not affect test 
performance, the data suggested a relationship between attitudes toward collaboration and use of 
Internet-based communication. It is encouraging that use of Internet-based communication 
increases the likelihood of completing the course activities, though no link by instructional theory 
was established between the two variables. It was speculated that those who used chat sessions 
and mailing lists were motivated by their peers, or self-motivated by their own engagement, but 
no measure of motivation was conducted in this study and thus the issue remains inconclusive. 

The results do not substantiate the remaining hypotheses. Therefore it may be concluded 
that in this population, attitudes toward collaboration are little effected by engagement in an 
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Internet-based course. In addition, using Internet-based communication such as chat sessions and 
mailing lists do not effect participant performance. Similarly, participants’ attitudes toward 
collaboration do not predict performance on Internet skill tests. 

Although not all results lead to positive conclusions, it is important to understand the 
aspects of Internet-based communication that may contribute to the development of collaboration 
among educators at a distance. Information about the characteristics of Internet-based 
communication and methods of implementing effective collaboration must be investigated to 
increase educators' effectiveness, to enhance achievement for all students and to promote 
systematic school improvement. This study provides a step toward such research. 

Collaboration is not a context-free concept: its implementation is tied to specific types of 
communication provided by the medium and the specific application within the medium. As 
indicated in this paper, mailing lists and chat sessions represent two different modes of 
communication. Within these modes, there is further classification by application such as 
moderated and unmoderated groups. Findings in this study should not be over-generalized and 
different research studies for different modes and applications of Internet-based communication 
are recommended. 

Due to the sample size and the distributional assumptions, variables in this study were 
examined in a pairwise manner. This research project serves as a preliminary study to identify 
relationships among variables. As relationships are identified, a coherent model specifying the 
inter-relationships among variables can be constructed. To achieve this goal, structural equation 
modeling with a large sample size will be employed. 
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